United States Patent and Trademark Ofhce 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



10/582,439 



FILING DATE 



06/09/2006 



7590 01/16/2009 

Blakely Sokoloff Taylor & Zafman 
12400 Wilshire Boulevard 
7th Floor 

Los Angeles, CA 90025 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



CONWAY, THOMAS A 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



KJttiVrXi nvrliyjts OUff Iff fcff Jr 


Application No. 

10/582,439 


Applicant(s) 

HU ET AL. 


Examiner 

THOMAS A. CONWAY 


Art Unit 
4182 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on &9/2006 . 
2a )□ This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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DETAILED ACTION 



It would be of great assistance to the Office if all incoming papers pertaining to a 
filed application carried the following items: 

1 . Application number (checked for accuracy, including series code and 
serial no.). 

2. Group art unit number (copied from most recent Office communication). 

3. Filing date. 

4. Name of the examiner who prepared the most recent Office action. 

5. Title of invention. 

6. Confirmation number (See MPEP § 503). 



Drawings 

1 . The drawings are objected to because box 4 of Fig. 1 contains indefinite 
language: "e.g.". Corrected drawing sheets in compliance with 37 CFR 1 .121(d) are 
required in reply to the Office action to avoid abandonment of the application. Any 
amended replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only one figure is being amended. The 
figure or figure number of an amended drawing should not be labeled as "amended." If 
a drawing figure is to be canceled, the appropriate figure must be removed from the 
replacement sheet, and where necessary, the remaining figures must be renumbered 
and appropriate changes made to the brief description of the several views of the 
drawings for consistency. Additional replacement sheets may be necessary to show the 
renumbering of the remaining figures. Each drawing sheet submitted after the filing date 
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of an application must be labeled in the top margin as either "Replacement Sheet" or 
"New Sheet" pursuant to 37 CFR 1 .121 (d). If the changes are not accepted by the 
examiner, the applicant will be notified and informed of any required corrective action in 
the next Office action. The objection to the drawings will not be held in abeyance. 



Specification 

2. The disclosure is objected to because it does not support the recited language of 

claims 9, 1 1 and 19. 

Regarding claims 9 and 19, the language of each which are similar, claim 9 recites, 
"and then modifying the classification of one or more of the pixels by considering spatial 
relationships between the locations of the classified pixels". The way the claim reads, it 
seems as if there is an initial classification, selection of threshold, binahzation and then, 
another reclassification based on the spatial relationships between pixels. There is no 
support for this in the specification. 

Regarding claim 1 1 , which recites, "at least one data input device for a user to 
select a region of interest in the image". Although the specification mentions a 
computer system, there is not explicit mention of an input device for the purposes of 
selecting a region of interest. 
Correction is required. See MPEP § 608.01(b). 
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Claim Objections 

3. Claim18 is objected to because of tine following informalities: recites 
"maximnise", examiner is assuming this is a misspelling of the word "maximise". 
Appropriate correction is required. 



Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 



5. Claims 1-4 and 9 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Farrokhnia et al. (US 6,694,047 B1), hereafter referred to as "Farrokhnia". 

Regarding claim 1, Farrokhnia discloses a method of binarising an image 
composed of pixels having respective intensity values, the method comprising: 
(i) using prior knowledge about the image to derive a region of interest within it (Col. 8, 
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lines 24-32); 

(ii) using prior knowledge about the image to derive an intensity range of pixels in the 
said region of interest (Col. 8, lines 44-50); 

(ill) obtaining a frequency distribution of the intensities within the said intensity range of 
pixels within the said region of interest (Col. 8, lines 51-60); 

(iv) using the said frequency distribution to derive an intensity threshold (Col., lines 22- 
23); and 

(v) binarising the image by classifying pixels in the said region of interest according to 
whether their intensities are above or below the said intensity threshold (Col. 9, lines 38- 
49). 

Regarding claim 2, Farrokhnia discloses a method in which the threshold is 

found by deriving a valley in the frequency distribution within the range, and selecting 
the intensity threshold to correspond to the valley (Col. 9, lines 30-31). 

Regarding claim 3, Farrokhnia discloses a method in which the valley is found 

by determining the total intensities in a number of intervals defined in the range (Col. 8, 
lines 53-57; See also Fig. 5), and selecting the intensity threshold as an intensity within 
the interval having the lowest total intensity (570 in Fig. 5). 
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Regarding claim 4, Farrokhnia discloses a method in which the intensity 
threshold is selected as the mid-point of the interval having the lowest total intensity 
(Col. 9, lines 30-31). 

Regarding claim 9, Farrokhnia discloses a method of processing an image 
which includes binarising it by a thresholding method according to claim 1 , and then 
modifying the classification of one or more of the pixels by considering spatial 
relationships between the locations of the classified pixels (Col. 9, lines 50-65). 

Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or deschbed as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. This application currently names joint inventors. In considering patentability of 
the claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of 
the various claims was commonly owned at the time any inventions covered therein 
were made absent any evidence to the contrary. Applicant is advised of the obligation 
under 37 CFR 1 .56 to point out the inventor and invention dates of each claim that was 
not commonly owned at the time a later invention was made in order for the examiner to 
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consider the applicability of 35 U.S.C. 103(c) and potential 35 U.S.C. 102(e), (f) or (g) 
prior art under 35 U.S.C. 103(a). 

8. Claims 10-14 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Farrokhnia, in view of Uppaluri et al. (US 6,466,687 B1 : "Uppaluri "). 

Regarding claim 10 and 11, while Farrokhnia discloses a processor arranged to 
obtain a frequency distribution of the intensities within the intensity range of pixels within 
the region of interest, use the frequency distribution to derive an intensity threshold; and 
binarise the image by classifying pixels in the region of interest according to whether 
their intensities are above or below the threshold (Col. 3, lines 1-4, See also Claim 1 as 
detailed previously in this office action), Farrokhnia fails to disclose a computer program 
product comprising a recording medium and programming instructions stored on the 
recording medium and readable by a computer system to cause the computer system to 
perform a method according to claim 1 and at least one data input device for a user to 
select a region of interest in the image and specify a frequency range within the 
frequency distribution of the intensities of pixels in the region of interest. 

Uppaluri discloses a computer program product (Fig. 13) and a data input device 
(1340, in Fig. 13) for a user to select a region of interest in the image (Col. 4, lines 45- 
55) and specify a frequency range within the frequency distribution of the intensities of 
pixels in the region of interest (Col. 6, lines 34-42). 

Farrokhnia's disclosure did not mention manual inputting of information needed 
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by his nnetliods to process the image, select a region of interest or specifying a 
frequency range related to pixels in the region of interest, even though there is mention 
of importation of the image to be processed. Farrokhnia's methods seem wholly 
automated, while Uppaluri makes mention that segmentation can be done automatically 
or manually (Col. 4, lines 45-47). 

Since Uppaluri's disclosure accommodates both a manual as well as automated 
setting, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in Farrokhnia's method, a computer program product to 
implement the segmentation task as well as an input device to perform the related 
actions needed to perform segmentation using a' priori knowledge in an efficient 
manner. 

Regarding claim 12, Farrokhnia discloses a processor (Col. 3, lines 1-4) 
arranged to derive a threshold found by deriving a valley in the frequency distribution 
within the range, and selecting the intensity threshold to correspond to the valley (Col. 
9, lines 30-31). 

Regarding claim 13, Farrokhnia discloses a processor (Col. 3, lines 1-4) 
arranged to find a valley by determining the total intensities in a number of intervals 
defined in the range (Col. 8, lines 53-57; See also Fig. 5), and selecting the intensity 
threshold as an intensity within the interval having the lowest total intensity (570 in Fig. 
5). 
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Regarding claim 14, Farrokhnia discloses a processor (Col. 3, lines 1-4) 
arranged to select the intensity threshold as the mid-point of the interval having the 
lowest total intensity (Col. 9, lines 30-31). 

Regarding claim 19, Farrokhnia discloses a processor (Col. 3, lines 1-4) 
arranged to process the segmented image by modifying the classes to which each pixel 
is allocated by considering relationships between the locations of the pixels which have 
been classified (Col. 9, lines 50-65). 

9. Claims 5-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Farrokhnia, in view of Ramesh at al. (Thresholding based on histogram approximation, 
lEE Proc.-Vis. Image Signal Processing, Vol 142, No. 5, pp 271-279, October 1995), 
hereafter referred to as "Ramesh". 

Regarding claim 5, Farrokhnia discloses the limitations of claim 1 but fails to 
disclose the limitations of claim 5. 

There are many known methods to find a threshold for a particular distribution of pixel 
intensities utilizing a histogram and its associated values, the Farrokhnia method does 
not speak on the actual manipulation of equations that was used to derive the threshold 
but makes mention that it is related to the variance (Col. 9, line 36). 
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Ramesh discloses a method where the threshold is found by minimising a function 
which is a sum of the variances of the intensities below and above the threshold (Sec. 
2.2.2). 

Since segmentation using histograms and related thresholds can be done using several 
methods, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to include in Farrokhnia's method, the steps used in Ramesh's 
disclosure as one other way to accomplish the same feat of deriving a threshold. 

Regarding claim 6, Farrokhnia discloses the limitations of claim 1 but fails to 
disclose the additional limitation of claim 5 upon which claim 6 is dependent. 
There are many known methods to find a threshold for a particular distribution of pixel 
intensities utilizing a histogram and its associated values, the Farrokhnia method does 
not speak on the actual manipulation of equations that was used to derive the threshold 
but makes mention that it is related to the variance (Col. 9, line 36). 
Ramesh discloses a method as in claim 5 where the sum of variances is weighted (Sec. 
2.2.2). Wi and W2 are arbitrary variables representing a weight applied to the sum and 
could be represented by "A" and "C" (Sec. 2.2.2, pg 274), which are associated with the 
original a' priori weights of Pi and P2 Sec. 2.2.2, pg 274). Weighing by class probability 
is a common statistical operation. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to include in the method as outlined by Farrokhnia the step of 
weighting the sum of variances as presented by Ramesh in order to improve the 
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segmentation process, referred to by Ramesli as tlie "goodness" of a tlireshold (Sec. 3, 
paragrapli 1). 



10. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Farrol^linia, in view of Ramesh and in further view of N. Otsu (A Tliresiiold Selection 
Method from Gray-Level Histograms, IEEE Transactions on Systems, Man, and 
Cybernetics, Vol SMC-9, No. 1, pp 62-66, January 1979), hereafter referred to as Otsu. 

Farrokhnia and Ramesh discloses the limitations of claim 6 upon which 7 is 
dependent but fails to disclose representing labeling the possible values of pixel 
intensity by an integer index i and their respective frequencies by h(i), and writing the 
lower and upper intensities respectively as r.sub.low and r.sub.high and their associated 
equations as recited in claim 7. The use of "i", "h(i)" and "r.sub.low", "r.sub.high" are 
arbitrary notations representing the pixel intensity index, their respective frequencies 
and placeholder for low and high intensities and could just as well be labeled "x" and 
"g(x)" or any other values. 

Otsu discloses a pixel intensity index "i" ("1", Page 63, paragraph 2) as well as 
"h(i)" ("ni", Page 63, paragraph 2), as well as "r.sub.low" ("1", Page 63, paragraph 2) 
and "r.sub.high" ("L", Page 63, paragraph 2). Other associated terms can also be found 
in Otsu's teachings: weighted sum (weighted by class probabilities. See EQ 14), class 
probabilities (See EQs 2 and 3), "D(c1)" and "D(c2)" (See EQ 13), and mean values 
(See EQs 7 and 8). 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made, to include in the method as outlined by Farrokhnia, supplemented 
by the teachings of Ramesh, the methods and notations as used by Otsu in order to 
maximize seperability of resultant class of pixels (Abstract) which would make the 
segmentation more efficient. 

1 1 . Claims 15-16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Farrokhnia, in view of Uppaluri and in further view of Ramesh. 

Regarding claims 15-16 Farrokhnia and Uppaluri disclose the limitations of 
claim 14 but does not teach a selecting the threshold by minimising a function which is a 
sum of the variances of the intensities below and above the threshold, the sum or which 
is a weighted sum based on two constants W.sub.1 and W.sub.2. 

Ramesh discloses a method where the threshold is found by minimising a 
function which is a sum of the variances of the intensities below and above the 
threshold (Sec. 2.2.2). Ramesh also discloses a method as in claim 5 where the sum of 
variances is weighted (Sec. 2.2.2). Wi and W2 are arbitrary variables representing a 
weight applied to the sum and could be represented by "A" and "C" (Sec. 2.2.2, pg 274), 
which are associated with the original a' priori weights of Pi and P2 Sec. 2.2.2, pg 274). 
Weighing by class probability is a common statistical operation. 

Since segmentation using histograms and related thresholds can be done using 
several methods, it would have been obvious to one of ordinary skill in the art at the 
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time the invention was made, to include in Farrol^linia and Uppaluri's method, the steps 
used in Ramesh's disclosure as one other way to accomplish the same feat of deriving 
a threshold as well as including the step of weighting the sum of variances as presented 
by Ramesh in order to improve the segmentation process, referred to by Ramesh as the 
"goodness" of a threshold (Sec. 3, paragraph 1). 

12. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 

Farrokhnia, in view of Uppaluri, in view of Ramesh and in further view of Otsu. 

Farrokhnia, Uppaluri and Ramesh teach the limitations of claim 16 upon which 
claim 17 is dependent, but fails to teach labeling the possible values of pixel intensity by 
an integer index i and their respective frequencies by h(i), and writing the lower and 
upper intensities respectively as r.sub.low and r.sub.high and their associated equations 
as recited in claim 7. The use of "i", "h(i)" and "r.sub.low", "r.sub.high" are arbitrary 
notations representing the pixel intensity index, their respective frequencies and 
placeholder for low and high intensities and could just as well be labeled "x" and "g(x)" 
or any other values. 

Otsu discloses a pixel intensity index "i" ("1", Page 63, paragraph 2) as well as 
"h(i)" ("ni", Page 63, paragraph 2), as well as "r.sub.low" ("1", Page 63, paragraph 2) 
and "r.sub.high" ("L", Page 63, paragraph 2). Other associated terms can also be found 
in Otsu's teachings: weighted sum (weighted by class probabilities. See EQ 14), class 
probabilities (See EQs 2 and 3), "D(c1)" and "D(c2)" (See EQ 13), and mean values 
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(See EQs 7 and 8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made, to include in the system as outlined by Farrokhnia and 
Uppaluri, supplemented by the teachings of Ramesh, the methods and notations as 
used by Otsu in order to maximize seperability of resultant class of pixels (Abstract) 
which would make the segmentation more efficient. 

13. Claim 8 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Farrokhnia, in view of Cheng et al. (Threshold Selection Based on Fuzzy c-Partition 
Entropy Approach, Pattern Recognition, Vol. 31, No. 7, pp 857-870, 1998), hereafter 
referred to as "Cheng". 

Farrokhnia discloses the limitations of claim 1 upon which claim 8 is dependent 
but fails to teach the limitations of claim 8. 

Cheng discloses selecting the threshold as a function of parameters which maximise an 
entropy function which indicates the entropy of a fuzzy partition of the pixels into classes 
based on the parameters (Sec. 5.1 ). 

As mentioned previously in the analysis of claim 5, there are many methods of 
segmenting using thresholding and selecting a threshold based on a fuzzy c-partition 
was among the methods at the time of the invention. Fuzzy c-partition ing is an effective 
method to select thresholds in an automatic environment (Abstract). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 



Application/Control Number: 10/582,439 Page 15 

Art Unit: 4182 

time the invention was made, to include in tlie metliod as outlined by Farrokhnia, the 
methods as presented by Cheng in order to efficiently select segmenting thresholds in 
an automated environment as Farrokhnia discloses. 

14. Claim 18 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Farrokhnia and Uppaluri, in view of Cheng. 

Farrokhnia and Uppaluri disclose the limitations of claim 11 upon which claim 18 
is dependent but fails to teach the limitations of claim 18. 

Cheng discloses selecting the threshold as a function of parameters which maximise an 
entropy function which indicates the entropy of a fuzzy partition of the pixels into classes 
based on the parameters (Sec. 5.1). 

As mentioned previously in the analysis of claim 5, there are many methods of 
segmenting using thresholding and selecting a threshold based on a fuzzy c-partition 
was among the methods at the time of the invention. Fuzzy c-partitioning is an effective 
method to select thresholds in an automatic environment (Abstract). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made, to include in the system as outlined by Farrokhnia and 
Uppaluri, the methods as presented by Cheng in order to efficiently select segmenting 
thresholds in an automated environment as Farrokhnia discloses. 
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Conclusion 

15. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: Erdogan et al. (US 6,567,771 B2) discloses weighing constants 
applied to variances to better separate confusable class pairs. 

16. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to THOMAS A. CONWAY whose telephone number 
is (571)270-5851 . The examiner can normally be reached on Monday through Friday 

BAM - 5PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Benny Tieu can be reached on 571-272-7490. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. 

For more information about the PAIR system, see http:llpair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (BBC) at 866-21 7-91 97 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
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automated information system, call 800-786-91 99 (IN USA OR CANADA) or 571 - 
272-1 000. 

/Thomas A. Conway/ 
Examiner, Art Unit 4182 



/Benny Q Tieu/ 

Supervisory Patent Examiner, Art Unit 4182 



